Repairing Concavities in ROC Curves
نویسنده
چکیده
Declaration This dissertation is submitted to the University of Bristol in accordance with the requirements of the degree of Bachelor of Science in the Faculty of Engineering. It has not been submitted for any other degree or diploma of any examining body. Except where specifically acknowledged, it is all the work of the Author. 3 ABSTRACT Machine Learning applications require learning algorithms that satisfy appropriate performance criteria. Refinements have been developed for the SwapCurve algorithm improving the performance of scoring classifiers by correcting inaccurate predictions. Analysis of theory underpinning the algorithm leads to two refinements, improving reliability of results. The two refinements have been implemented for naïve Bayes and RIPPER and performance no longer declines for any dataset in this research. The key overall achievement is a four-fold increase in average performance gain for both learning algorithms. A suite of Java software, developed and applied successfully to achieve the results, is now available to assist further research.
منابع مشابه
Repairing Concavities in ROC Curves
In this paper we investigate methods to detect and repair concavities in ROC curves by manipulating model predictions. The basic idea is that, if a point or a set of points lies below the line spanned by two other points in ROC space, we can use this information to repair the concavity. This effectively builds a hybrid model combining the two better models with an inversion of the poorer models...
متن کاملShape recognition: convexities, concavities and things in between
Previous studies on shape recognition have drawn different conclusions regarding the importance of specific object features, such as convexities, concavities and intermediate points. Some studies found evidence for a predominant role of convexities, whereas others favored concavities or intermediate parts. However, most studies have employed familiar objects or simple geometric shapes not neces...
متن کاملROC curves and nonrandom data ∗ Jonathan Aaron
This paper shows that when a classifier is evaluated with nonrandom test data, ROC curves differ from the ROC curves that would be obtained with a random sample. To address this bias, this paper introduces a procedure for plotting ROC curves that are inferred from nonrandom test data. I provide simulations and an example with wine data to illustrate the procedure as well as the magnitude of bia...
متن کاملSemiparametric Inferential Procedures for Comparing Multivariate Roc Curves with Interaction Terms
Multivariate ROC curve models that include an interaction term between biomarker type and false positive rate are important in comparative biomarker studies, because such interaction allows ROC curves of different biomarkers to cross each other. However, there has been limited work in drawing inference for comparing multivariate ROC curves, especially when interaction terms are present. In this...
متن کاملMixtures of receiver operating characteristic curves.
RATIONALE AND OBJECTIVES Receiver operating characteristic (ROC) curves are ubiquitous in the analysis of imaging metrics as markers of both diagnosis and prognosis. While empirical estimation of ROC curves remains the most popular method, there are several reasons to consider smooth estimates based on a parametric model. MATERIALS AND METHODS A mixture model is considered for modeling the di...
متن کامل